Hybrid Block-Based/Segment-Based Video Compression at very Low Bitrate

In this paper, an algorithmic architecture of video compression at very low bit rate is presented. This architecture incorporates block-based as well as segment-based coding algorithms and manages them in a cooperative way in which different approaches are applied to different image frames iteratively: I and P frames are coded by block-based algorithm and 0 frames are coded by segment-based algorithm. An extra degree of freedom in video coding is introduced in order to achieve higher compression ratios while maintaining good visual quality. Every other frame in the image sequence is coded using block-based coding (ITU-T H.263) with motion (block-matching, 2D translations) and color (DCT) coefficients. These I and P frames are allocated with more bits, generating images with higher spatial accuracy, and providing the coder with generic coding capability. Between every two P frames, the 0 frame is coded using segment-based algorithm providing the motion (rigid 2D source models, 3D motions) and shape (polygon or spline approximation) informations. Since only motion prediction is provided, pixel values are copied or interpolated from neighboring P frames. With good segmentation and affine transform, the prediction can preserve the natural shapes of the objects and give natural motion information which is sufficient for very low bitrate coding. These frames are allocated with less bits, generating images with less temporal accuracy, but providing the coder with higher compression efficiency. For each 0 frame, the image can be divided into three parts: static background, moving objects, and uncovered background. Moving objects are the only parts that encoder generates output to the bitstream. Each moving object is encoded by its shape and motions (forward and backward). Shape is coded as a set of vertices in a polygon (or spline) approximation. Motion is coded as a set of coefficients that represent an affine transform: S(X, Y) = S’ ( a,X+aZY+a3 a,X+a5Y+a6 a7X + UgY + 1’ a7X + a,Y + 1 1 fmv,bmv At the decoder side, the uncovered background is obtained by applying the inverse motion to the object. The value of the uncovered background is found in the frame which is not involved in estimating the motion. For example, uncovered background caused by forward motion, which is estimated from frame n-l and n, is filled with values in frame n+l Simulations are done for MPEG-4 Class A test sequences which contain low spatial detail and low amount of movement but are not restricted to head-and-shoulder images. The results are encouraging in that the coder provides higher image quality with satisfactory motion effect. 1068-0314/96$5.00